Refactor Reports by Block and Services and enhancements #62
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ervices`, `staked_apps_by_block_and_services`, and `relay_by_block_and_services`, replacing unique index and upsert constraints to avoid unnecessary overhead. - Implemented retry utility for handling module accounts and integrated it into `handleModuleAccounts`. - Enhanced block report generation by running database functions in parallel for performance improvements. - Refined block processing in `handleBlock` to include timing logs for profiling purposes.
Alann27
force-pushed
the
refactor-reports-by-block-and-services
branch
from
4d470ae to
20796dd
Compare
jorgecuesta
approved these changes
Sep 25, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
staked_suppliers_by_block_and_services,staked_apps_by_block_and_services, andrelay_by_block_and_services, replacing unique index and upsert constraints to avoid unnecessary overhead.handleModuleAccounts.handleBlockto include timing logs for profiling purposes.Issue
In beta, we have more than 10,000 services. This caused the indexer to take over 30 seconds to index a single block due to the heavy load from reports by block and services.
The tables
staked_suppliers_by_block_and_servicesstaked_apps_by_block_and_servicesrelay_by_block_and_serviceswere being created through SubQL (since they were defined in
schema.graphql). Because historical data was enabled, these tables included the_block_rangecolumn, and SubQL automatically generated compound indexes for each foreign key combined with_block_range.However, historical data does not apply to these tables, since we insert new records for every block. This means we don’t need the
_block_rangecolumn or the additional indexes created by SubQL.With this change, the indexer now takes around 5 seconds to index a block (down from 30+ seconds).
We also encountered an issue when fetching module accounts: queries sometimes returned no results for the block being indexed. To address this, we added a retry mechanism that keeps querying for module accounts until they become available or the retry limit is reached.
Type of change
Select one or more:
Sanity Checklist